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ABSTRACT 

Intended for administrators and policy makers as well 
as teachers, this digest identifies for prospective purchasers 
various sources that offer reviews of educational computer software 
for English and the language arts. Following an introduction, the 
first section of the digest discusses content-specific as well as 
general educational computing subscription publications. The next two 
sections examine help available from professional associations and 
consortia services. The fourth section explores online sources for 
titles of recommended software, while the fifth discusses sources for 
published catalogs of approved software. The final section covers 
informal sources, such as independent distributors and consultants. 
(HTH) 
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HOW TO FIND GOOD COMPUTER SOFTWARE IN ENGLISH AND LANGUAGE ARTS 
Anne Auten 

One of the great sports in the field of educational computing is 
called ••find the good software," particularly in the field of 
English, reading, and the other language arts. Many English 
teachers , spurning the current unsatisfactory software developed 
for computer assisted instruction, have turned away from CAI and 
toward the use of the computer as a tool for teaching word 
processing in composition instruction and for computer database " 
searching in units on library skills. 

The ease of developing lower level computer software and the 
difficulty of programing some areas of the language arts 
curriculum are largely responsible for the less-than-ideal 
quality of software currently available • Because many aspects of 
language arts involve mental and verbal processes not adapted 
easily to computer-delivered instruction, software developers 
have been producing programs that, for the most part, are drill 
and practice lessons on aspects that are more easily defined: 
parts of speech, spelling, vocabulary, punctuation, and syntax. 

In a sense, the problem of finding good software is further 
complicated by the fact that some good CAI materials have 
appeared on the market-~for example, tutorials that deal with the 
cloze procedure in teaching reading, sentence combining, and 
general comprehension* Educators now need the ability to evaluate 
software knowledgeably and the resources that provide thorough, 
up-to-date software reviews. For a brief discussion of evaluating 
language arts software, see the ERIC digest, "Software Evaluation 
for the Teacher of English Language Arts*" This particular digest 
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identifies for prospective purchasers various sources that offer 
reviews of language arts software. 

REVIEWS IN SUBSCRIPTION PUBLICATIONS 

The most readily available resoui'ce for software reviews is 
subscription publications. Englisih and language arts educators 
now have a content-specific journal that reports software 
applications and evaluations in Hheir discipline. Computers , 
Reading and Language' Arts is a quarterly journal that features 
extensive reviews written by computer-using educational 
specialists. Each CRLA courseware evaluation follows a prescribed 
format that includes the program sequence, the program's 
educational intent and content, a discussion of the instructional 
technique(s) used by the program, and a description of any 
support literature or documentation provided by the publisher. 

In addition to CRLA, English and language arts specialists 
have another content-specific resource in The Computing Teacher 
which, since it began publishing, has featured a monthly column, 
"Computers in the Teaching of English,** edited by Robert Shostak 
and Lester Golub. English teachers are invited to submit reports 
on their use of computers in the classroom and to describe 
applications of particular software packages. 

English and language arts specialists have found the monthly 
software reviews in other educat ional comput ing journals such as 
Electronic L pai'n ing , Electronic Education and Teaching and 
Computers co be a helpful resource. In addition to reviews of 
software designed for their discipline, they look at materials 
developed for other areas such as social studies, business 
communication, or word problem solving in mathematics. These 
programs often can be used to advantage as discussion starters 
for prewriting sessions or as exercises in logical thinking or 
persuasive argument in a language arts setting. Electronic 
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Education has presented an "Annual Buyer's Guide" for the last 
three years, an annotated list of programs grouped according to 
content area designed to assist educators with end-of-year 
purchasing decisions. The May/June 1984 issue of Teaching aqd 
Computers offered a pull-out supplement that briefly described 
more than 70 programs recommended by teacners. The latter 
journal, published by Scholastic^ Inc. for eleir.entary-level 
educators, also carries a regular feature called "Softwa.re 
Showcase: Software Recommended for Teachers by Teachers," that 
provides one- or two-paragraph evaluations. In addition, each 
issue off ers' a detailed discussion of a "Program of the Month," 
complete with a listing of the programming code. These 
noncommercial programs have been designed by teachers who make 
them available to others interested in computer-assisted 
instruction. Even such general computing journals as Personal 
Software: The Monthly Review of the Best Packages offer 
'extensive reviews of a selection of educational software packages 
in each issue. 

While not educational computing journals per se, three other 
subscription publications offer extensive reviews of educational 
software, and would serve well those teachers looking for good 
language arts software. The first is EPIE and Consumers Union 
Microcomputer Courseware PRO/FILE S, a set of 8 1/2" x 11" file 
cards and file box, "omplete with subject matter dividers. 
Software in subject matter areas that could be effective in the 
English/language arts classroom include The Arts, Business 
Education, Computer Literacy, Language Arts, Logic/Problem 
Solving, Reading, and Social Studies. These headings are 
indicated in bold print at the top of each evaluation card, 
making them easy to file and find. Each monthly update of cards 
offers subscribers a summary and in-depth evaluation of 30 
individual software packages distributed across the curriculum. 
The discussion of each package has four major strands: goals and 
objev^tives, contents^ methods and approach, and evaluation and 
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managements Next is Coursew are Report Card, a publication 
^ . — 

available in both elementary and secondary editions that presents 
two- to four- page summary/evaluations of 25-30 packages per issue. 
Users of Apple, Atari , Commodore, and Radio Shack microcomputers 
can subscribe to separate editions. 

Finally, for those who have neither the time nor the money* 
to subscribe to several educational computer journals, The Digest 
' 'lOf Software Reviews; Education provides abstracts and indices of 
reviews from over 60 publications in the U.S* and Canada* A 
software program must have had a minimum of two published reviews 
to be selected for inclusion in The Digest s 

Keep in mind that reviews of specific microcomputer 
brand-compatible programs are also published in journals by 
various hardware manufacturers, Apple computer's Journal of 
Courseware Review i s an example • 

HELP FROM PROFESSIONAL ASSOCIATIONS 

While neither of the two major support organizations for teachers 
of English, reading, and the other language arts — the National 
Council of leachers of English (NCTE) and the International > 
Reading Association (IRA)— regularly publishes '^software reviews, 
they ai^e involved and interested in microcomputer u^sage in their 
content area classrooms, NOTE'S Committee on Instructional 
Technology has published guidelines for evaluating language arts 
software; IRA's Committee on Technology and Reading has produced 
Guidelines for Educators designed to help reading teachers make 
the best possible use of the new technologies. Both associations 
publish journals that carry articles recommending various 
software programs. For example, the IRA journal. The Reading 
Teache r, carries a monthly column called "Printout" with CAl 
suggestions for elementary reading teachers. Apart from NCTE and 
IRA# the National Education Association supports an . Educational 
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Computer Service that publishes a catalog of "NEA Teacher 
Certified" software. The software has been evaluated and approved 
by trained progranuaers and teachers using specific guideline^' , 

CONSORTIA SERVICES 

At both national and state levels, computer-oriented educational 
consortia have been organized to offer educators such services as 
teacher inservice workshops, "help" hotlines for technical 
assistance, hardware acquisition, and software reviews and 
recommendations . 

State and local consortia often provide, as one of their 
services, a library of software packages for teachers to preview. • 
The January and February 1984 issues of Electronic Learning 
include a directory listing the location, size of inventory, and' 
contact person for each state's noncommercial preview centers, 
many of which have been developed by state-supported regional 
consortia, as well as university labs, individual school 
districts, and educational associations. At the national level, 
the Educational Software Evaluation Consortium/ representing 27 
organizations involved in computer education throughout North 
America, has developed a list of favorably reviewed " instructional 
software for K-12 classrooms. 

ONLINE SOURCES 

Us«-ing a microcomputer and a modem, educators can locate titles of 
recommended software by searching commercial information 
databases. The microSIFT Reviews ^ prepared by the Northwest 
Regional Educational Laboratory's Computer Technology Program, 
are available in the Resources in Computer Education (RICE) 
database and in the ERIC database. The reviews are also available 
in print editions through regional and local educational service agen- 
cies. The format of a microSIFT Review is a page of comments in the 
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areas of instructional ob jective.s, instructional prereq iisites, 
content and structure, potential uses, major strengths and 
w^^knesses, and a 21-itein checklist from the microSIFT software 
evaluation' form. ' . 

•Both the J.ICIE and the ERIC files are provided by 
Bibliographic Retrieval Services (BRS)* and are available to those 
who have a search contract with BRS in Latham, New York. The ERIC 
file is also a ailable through the DIAi^OG Information Retrieval 
Service, as is the Microcomputer Index file. The Microcomputer 
Index is a subject and abstract guide to magazine articles from 
fi over 40 microcomputer journals, and includes software reviews 
published in those journals. As with BRS, DIALOG users are 
assigned a password in their search contract and are billed* only 
for the time they use the database. 

PUBLISHED CATALOGS 

Facing the plethora of catalogs produced by software and hardware 
companies can be overwhelming, in addition to the catalogs 
published by manufacturers, individual distributors list software 
they claim is "teacher-tested." Companies such as Scholastic, 
Hammett, and the Society for Visual Education (SVE) produce 
catalogs listing what they describe as educator-evaluated and 
approved software. Most of these distributors are willing 
to offer a free examination and return policy* 

When searching for potential program titles for English and 
language a^rts, from any catalog, educators should not hesitate to 
examine software descriptions in content areas other than English 
language arts. Shirley Keran, language arts software designer for 
\he Minnesota Educational Computing Consortium (MECC), suggests 
that many programs exist for other disciplines that can augment 
the work of the language arts teacher. If an English teacher 
believes that world events can be the topic for an essay^ then 
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software developed for social studies could be applicable. "The 
epicenters of earthquaker> and the waterways of voyageurs can be 
simulated, studied, and researched" says Keran, "Many simulations 
could be used in social studies for one set of activities and in 
language arts for another." 

INFORMAL SOURCES 

A final suggestion for educators interested in locating English 
and language arts software is to identify and befriend an 
independent distributor of software. A growing number of 
computer-using teachers are becoming distributors for companies 
whose products they have used successfully with their students* 
Many of these enthusiastic entrepreneurs attend trade shows, 
preview the latest software releases, and in general keep current 
with both software and hardware developments. As a rule, they are 
willing to consult with educational administrators interested in 
introducing CAI to their staffs, to offer inservices on software 
applications in the various content areaS; and to make new 
softwari=^ releases available for review. ' , 

All of the sources described in this digest are suggested 
only to assist educators in locating software for preview; 
''because the criteria used for evaluation differ from one source 
to another; no qualitative review, no matter how positive it 
sounds, should be taken as ^n endorsement to purchase and use" a 
piece of software without a preview by those who will be its 
ultimate users. 
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Sources for Software Reviews 
Subscription Publications 

Computers y Reading a n d Language Arts , Modern Learning Publishers, 
Inc., 1308 East 38th Street, Oakland, CA 94602, $14/year. 
m> ■ . 

Courseware Report Card , Educational Insights, 150 West Carob Street, 
Compton, CA 90220, $59.50/year. 

Digest of Software Reviews; Education > 1341 Bulldog Lane, Suite C, 
Fresno, CA 93710, $52.95/year. 

Electronic Education , Electronic Communications, Inc., Suite 220, 
1311 Executive Center Drive, Tallahassee, PL 32301, $18/year, 

Electroniq Learning , Scholastic Inc., 730 Broadway, New York, NY 
10003-9538, $19/year. 

Journal of Courseware Review , Apple Computer, 20525 Mariana Aveniie, 
Cupertino, CA 95014, $5.95/issue. 

Micro-Courseware PRO/FILES. EPIE and Consumers Union, P.O. Bo?c 839, 
Watermill, NY 11976, $125/year, including, a one-year sub- 
scription to The Computing Teacher ^. 

Personal Software , P.O. Box 2919, Boulder, CO 80323, '$24/year. 

Teaching and Computers . Scholastic Inc., 730 Broadway, New York, 
NY 10003-9538, $19/year. 
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The Computing Teacher # The International Council for Computers in 
Education (ICCE), University of Oregon, 1787 Agate Street, 
Eugene, OR 97403-1923, $21.50/year. 

Online Sources 

Bibliographic Retrieval Services, Inc., 1200 Route 7, Latham, NY 
12110. 

«.* 

DIALOG Information Services, Inc., 3460 Hillview Avenue, Palo Alto, 
CA 94304. 

microSIFT Reviews, Northwfest Regional Educational Laboratory, 300 
S.W. Sixth Avenue, Portland, OR 97204. 

Specific Guides 

"A+ Software Supplement," Teaching and Computers ^ 1 (May-June 1984): 
29-24. 

•J ♦ 

International Reading Association Committee on Technology and Reading. 
) "Guidelines for Educators." Newark, Del.: IRA, 1984. 

National Council of Teachers of English Coimnittee on Instructional - 
. Technology. "Guidelines for Review and Evaluation of English 
Language Arts Software." Urbana, 111.: NCTE, 1983. 

(J 

National Education Association Educational Computer Jervice. The 
Yellow Book of Computer Products for Education . Bethesda , 
Md.: NEA, 1984. 

"Third Annual Buyer's Guide," Electronic Education , 3 (March 1984): 
25-48. 
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HOW TO FIND GOOD COMPUTER SOFTWARE 
IN ENGLISH AND LANGUAGE ARTS 



One of the great sports in the field • aducational computing is 
called "find the good software," ^ irticularly in the field of 
English, reading, and the other laiijuage arts. Many English 
teachers, spurning the current unsatisfactory software devel- 
oped for computer-assisted instruction, have turned away from 
CAl and loward the use of the computer as a tool for teaching 
word processing in compq^sition instruction and for computer 
database searching in units on library skills. 

The ease of developing lower level computer software and 
the difficulty of programing some areas of the language arts 
curriculum are largely responsible for the less-than-ideal qual- 
ity of software currently available. Because Wany aspects of 
language arts involve mental and verbal processes not adapled 
easily to computer-delivered instruction, software developers 
have been producing programs that, for the most part, are drill 
and practice less(5ns on aspects that ar^ more easily defined, 
parts of speech, spelling, vocabulary, pun(^uation. and syntax. 

In a sense, the problem of finding good software is further 
complicated by the fact that some good CAl materials have 
appeared on the market -for example, tutorials that deal with 
the cloze procedure in teaching reading, sentence combining, 
and general comprehension. Educators now need the ability 
to evaluate software knowledgeably and the Resources that 
provide thorough, up-to-date software reviews. For a brief dis- 
cussion of evaluating language arts software, see the ERIC 
digest. "Software Evaluation for the Teacher of English Lan- 
guage Arto." This particular digest identifies for prospective 
purchasers various sources that offer reviews of language arts 
software. 

Reviews in Subscription Publications 

The most readily available resource for software reviews is 
subscription publications. English 3nd language arts educators 
now have a content-specific journ:il that reports software appli- 
cations and evaluations in their discipline. Computers. Reading 
and Language Arts is a quarterly journal that fealv ^es exten- 
sive reviews written by computer-using educational specialists. 
Each CRLA courseware evaluation follows a prescribed format 
that includes the program sequence, the program's educational 
intent and content, a discussion of the instructional tech- 
nique(s) used by the program, and a description of any support 
literature or documentation provided by the publisher. 

In addition to CRLA. English and language arts specialists 
have another conterjt-specific resource in The Computing 
Teacher which, since it began publishing, has featured a 
monthly coluijnn, "Computers in the Teaching of English." 
edited by Robert Shostak and Lester Golub. English teachers 
are invited to submit reports on their use of computers in the 
classroom and to describe applicatioris of particular software 
packages. 

English and language arts specialists have found the month- 
ly software reviews in other educational computing journals 
such as Electronic Learning. Electronic Education, and Teach- 
ing and Computers to be a helpful resource. In addition to 
reviews of software designed for their discipline, they look at 
materials developed for other areas such as social studies, 
business communication, or word fJroblem solving in mathe- 
matics. These programs often can be used to advantage as 
discussion starters for prewriting sessions or as exercises in 
logical thinking or persuasive argument in a language arts set- 
ting. Electronic Education has presented an "Annual Sugar's 
Guide" for the last three years, an annotated list of programs 
grouped according to content area designed to assist educators 
with end-of-year purchasing decisions. The May/June 1984 
issue of Teaching and Computers offered a pull-out supple* 
^ment that briefly described more than seventy programs recom- 
mended by teachers, The latter journal, published by Scliolas- 
tic. Inc. for tiementary-levdl educators, also carries a regular 
feature called "Software Showcase: Software Recommended 
for Teachers by Teachers," that provides one* or two-paragraph 
evaluations. In addition, each issue offers a detailed discussion 
of a "Program of the Month," complete with a listing of the 
programming code. These noncommercial programs have been 
designed by teachers who mak^ them available to others 
interested in computer-assisted instruction. Even such general 
computing journals as Personal Software: The Monthly Review 
of the Best Packages offer extensive reviews of a selection of 
educational software packages in each issue. 

While not educational computing journals per se, throe 
other subscription publications offer extensive reviews of edu- 



'cational software, and would serve well those teachers looking 
^or good language arts software. The first is EPI8^ and Con- 
sumers Union Microcomputer Courseware PRO/PILES, a set 
of 8'/?" X 11 " file cards and file box, complete with subject 
matter dividers. Software in subject matter areas that eould be 
effective in the English/language arts classroom nclude The 
Arts, Business Education, Computer Literacy. Language Arts. 
Logic/Problem Solving, Reading, and Social Studies. These 
headings are indicated in bold print at the top of each evalua- 
tion card, making them easy to, file and find. Each monthly 
update of cards offers subscribers a summary an^l in-depth 
evaluation of thirty individual software packages distributed*^ 
across the curriculum. The discussion of each package hasyf^Ur 
major strand^: goals and objectives, contents, methods and 
approach, and evaluation and management. Next is Course- 
ware Report Card, a publication available in both elemen- 
tary and secondary editions that presents two- to four-paye 
summary/evaluafToris of twenty-five to thirty packages per 
issue. Users of Apple. Atari, Commodore, and Radio Shack 
microcomputers carvsubscribe to separate editions. ' 

Finally, for thoi^e who have neither the time nor the money 
to subscribe to several educational computer journals. The 
Digest of Software Reviews: Education provides abstracts and 
indexes of reviews from over sixty publications in the U.S. atid 
Canada. A software program must have had a minimum of two 
published reviews to be selected for inclusion in The Digest. 

Keep in mind that reviews of specific microcomputer brand- 
compatible programs are also published in journals by various 
hardware manufacturers. Apple computer's Journal of Course- 
ware Review is an example. 

Help from Professional Associations 

WhHe neither of tha^ two major support organizations for 
teechers of English, reading, and the other language arfs-^ 
the National Council of Teachers of English (NCTE) and the 
International Reading Association (IRA}~-regularly publishes 
software reviews, they are involved and interested in micro* 
computer usage in their content area classrooms. NCTE's 
CorVimittee on Instructional Technology has published guide- 
lines for evaluating language arts software; IRA's Committee 
on Technology and Reading has produced Guidelines for Edu- 
cators designed to help reading teachers make the best possible 
use of the new technologies. Both associations publish jour- 
nals that carry articles recommending various software pro- 
grams. For example, the IRA journal. The Reading Teacher. 
carries a monthly column called "Printout" with CAl sugges- 
tions for elementary read'ng teachers. Apart frnm NCTE and 
IRA, the National Education Association supports an Educa- 
tional Computer Service that publishes a cjtalog of "NEA 
Teacher Certified" software. The software hiis been evaluated 
and approved by trained programmers and teachers using 
specific guidelines. 

Consortia Services 

At both national and state levels, computer-oriented educa- 
tional consortia have been organized to offer educators such 
services teacher inservice workshops, "help" hotlines for 
technical .ssistance. hardware acquisition, and software re- 
views and recommendations. 

^^tate and local consortia often provide, as one of their ser- 
vices, a library of software packages for teachers to preview. 
The Januarv and February 1984 issuos of Electronic Learning 
include a directory listing the location, size of inventory, and 
contact person for each state's noncommercial preview cen- 
ters, many of which have been developed by state-supported 
regional consortia, as weH js university labs, individual school 
districts, and educational associations. At the national leVel, 
the Educational Software Evaluadon Consortium, representing 
twenty-seven organizations involved in computer education 
throughout North America, has developed a list of favorably 
reviewed instructional software for K-1 2 classrooms. 

Online Sources 

Using a microcomputer and a modem, educators can locate 
titles of recommended software by searching commercial infor- 
mation databases. The microSIPT Reviews, prepared by the 
Northwest Regional Educational Laboratory's Computer Tech- 
nology Program, are available in the Resources in Computer 
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Education (RiCE) database and in the ERIC database. The 
reviews are also available in print editions through regional and 
local educational service agencies. The format of a micoSIFT 
review is a page of comments in the areas of instructional objec- 
tives, instructional prerequisites. con.tent dhd structure, poten- 
tial uses, major strengths^nd weaknesses, and a twenty-one- 
item checklist from the microSIrT software evaluation form. 
' "Both the RICE and the ERIC files are provided by Biblio- 
graphic Retrieval Services (BRS) and are available to those 
who have a search contract with BR5 in Latham. New York. 
The ERIC file is also available through the DIALOG Information 
Retrieval Service, as is the Microcomputer Index file. The Micro- 
computer Index Is a subject and abstract guide to magazine 
articles from over forty microcomputer journals, and includes 
software reviews published in those journals. As with BRS, 
DIALOG users are assigned a password in their search con- 
traet and are billed only for the time they use the database. 

Published Catalogs 

Facing the plethora of catalogs produced b/»software and 
hardware companies can be overwhelming. In addition to the 
catalogs published by manufacturers, individual distributors list 
software they claim is "teacher-tested." Companies sTuch as 
Scholastic, Hammett. and the Society for Visual Education 
(SVE) produce catalogs listing what they describe as educator- 
evaluated and a()proved software. Most of these distributors 
are willing to offer a free examination and return policy. 

When searching foV potential program titles for English and 
language arts from any catalog, educators should not hesitate 
to examine software descriptions in content areas other than 
English language arts. Shirley Keran. language arts software 
designer for the Minnesota Educational Computing Consortium 
(MECC). suggests that many programs exist for other disciplines 
that can augment the work of the language arts teacher. If an 
English teacher believes that world events can be the topic for 
an e^say. then software developed for social studies could be 
applicable, "The epicenters of earthquakes an'^ the waterways 
of voyageurs can be simulated, studied, and researched" says 
Keran. "Many simulations could be used In social studies for 
one set of activities and in language artp for another." 

Informal Sources 

A final suggestion for educators interested in locating English 
and language arts software is to identify and befriend an 
independent distributor of software. A growing number of 
Computer-using teachers are becoming distributors for co.n- 
panies whose products they have used successfully with their 
students. Many of these enthusiastic entrepreneurs attend 
trade shows, preview t.'.e latest software releases, and in 
general keep current with both software and hardware devel- 
opments. As a rule, they are willing to consult with educa- 
tranal administrators interested in introducing CAl to their 
staffs, to offer inservices on software applications in the various 
content areas, and to make new software releases available 
for review. 

All of the sources described in this digest are suggested 
only to assist educators in locating software for preview; 
because the criteria used for evaluation differ from one source 
to another, no qualitative review, no matter how positive i\ 
sounds, should be taken as an endorsement to purchase and 
use a piece of software without a preview by those who will be 
its ultimate users. 



Sources for Software Reviews 
Subscription Publicaticns 

Computers, Roading and Language Arts, Modern Leafniny PuolisherS. 

Inc., 1308 East 38th Street. Dakland. CA 94602. $14. year 
Courseware Report Card. Educational Insights. 1 50 Wost Cnrob Street. 

Compton. CA 90220. $59.5O/y0ar. 
Digest of Software Reviews Education. 1341 Bulldou Lane. Suite C. 

Fresno. CA 93710. S52.95/year. 
Electronic education, Electronic Communications. Inc. Suite 220. 

1311 Executive Center Drive, Tallahassee, FL 32301. $18 year 
Electronic Learning Scholastic Inc.. 730 8roadway. Now York. NY 

10003-9538. $10/ye8r. 
Journal of Courseware Review. Apple Computer. 20525 Manana 

Avenue. Cupertino. CA 95014. 85.95/issub. 
Micro-Courseware PRO /FILES. ERIE and Consumers Union. P 0 Box 

.839. Watermill. NY 1 1976. $125/year. including a one-year sub- 
scription to The Computing Teacher 
Personal Software, P.O. Box 2919. Boulder. CO 80322. $24/year 
Teaching and Computers. Scholastic Inc.. 730 Broadway. New York. 

NY 10003-9538. 019/year 
The Computing Teacher, The International Council for Computers in 

Education (ICCEI University of Oregon. 1 787 Agate Street. Eugene. 

OR 97403^4923. $2l .50/yedr. 

Online Sources 

Bibliographic Retrieval Services. Inc ^'^00 ftouto 7. Latham. NY 
12110. 

DIALOG Inforr^^ation Services. Inc., 3460 Hillview Avenue. Palo Alto. 
CA 94304. 

m/croS/f 7 Reviews. Northwest Regional Educational Laboratory. 300 
S.W. Sixth Avenue. Portland. OR 97204. 

Specific Guides 

"A+ Software Supplement," Teat,hing and Computers. 1 (Vay-June 
1984): 29-44. 

International Reading Association Committed on Technology and 
Reading. "Guldf.-llnes for Educators. " Newaik. Del. IRA. 1984 
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Large Scale Writing Assessment 



A national concern over the decline In students' writing scores 
(as revealed in National Assessment of Educational Progress 
reports), serious doubts about what some of those scores 
signify, and a shift In focus from writing product to writing 
process In research and classroom practice have each given 
Impetus to the change from indirect measures of writing pro- 
ficiency (those that use "objective" test Items) to direct mea- 
sures (those that call for studeni writing samples). In their 
'981 national survey, McCready and Meltpn found that of the 
twenty-four staiefrxialmlng to have a writing assessment pro- 
gram, twenty-two require a writing sample as part of .the assess- 
ment. Only two states rely solely on the use of objective tests. 

Large scale writing assessments, however. Involve a number 
of complex Issues that are not always evident to decision- 
makers who are not specialists In meesurement. In discussing 
how to and how not to conduct an assessment of student 
writing, McCaig (1982) warns that "an assessmisnt plan which 
Is Incomplete or poorly conceived may produce findings which 
can be challenged end even dismissed as meaningless by 
critics who can document flaws In the process." T^ls digest 
(1) outlines some of the approaches used in the impiementa- 
tlon of large scale writing assessments, (2) examines some of 
the issues and problems suirounding the use of student writ- 
ing samples, and (3) reports on various trends In state writing 
assessment projects. 

Direct Versus Indirect Assetsment 

Direct and indirect writing assessments are radically different 
approaches focusing on different components of writing. In- 
direct measurements typically use multiple choice tests to 
assess the student's understanding of mechanics or language 
conventions: spelling, punctuation, c'apltallxatlon, grammar, 
usage, sentence construction, organization, and so on. Direct 
assessments, on the other hand, assess actual writing perfor- 
mance, since they require the students to produce a writing 
sample. Spandel and Stiggins (1981) suggest that the two 
approaches can best be compared in terms oif their advantages 
and disadvantages. 



Direct 



Indirect 



Advantages 

• extent of informs 
tion provided about 
the students' writ- 

-—ingjiroficiMcy 

• fidelity to real world 
writing tasl(S 

• potential for posi- 
tive user attitudes 

• relatively low test 
development cost 

• high face validity 

• high score reliability 



• relatively low scor- 
ing cost 

• high degree of con- 
trol over sidlls 
tested 



Disadvantages 

• potahtiai lack of 
uniformity regard- 
ing proficiencies 

1 assessed - i^r 

• high cost of scoring 



• lacl< of fidelity to 
real world writing 
tasl(S 

• reliance on reading 

• lack of face validity 



Participants at a conference on assessment issues agreed that 
the use of writing samples is essential because of the instruc- 
tional Implications (McCready and Melton 1981). That Is, if 
teachers know that students' writing ability will be evaluated 
by means of a direct measure, they will encourage more writ- 
ing in the classroom. 

Three Approaches to Scoring 

Provided that writing assessments are conducted to determine 
the status of writing in a given state or school system and pro- 
vided they are conducted to help Improve classroom instruc- 
tion, several factors need to be considered prior to the collection 
of writing samples: (1) the educational decisions to be made, 
on the basis of test results; (2) the writing purpose, tfudlence, 
and type of writing to be required; and (3) the spedfic skills or 
traits to be judged along with the criteria used for evaluating 



writing performance (Spandel and Stiggins 1961), It must also 
be remembered that ratings will vary depending upon the scor- 
ing procedure used. Quollmalz, In writing about scoring criteria 
(1982), notes that "criteria employed for evaluating student 
writing vary along a number of dimensions: from qualitative to 
quantitative; froiM generaljo specific; from comprehensive, full 
discou.se features to Isolated features; from vague guidelines 
to repllcable, objective guidelines," Scoring options range from 
holistic scoring (general Impressionistic marking) to analytic 
and primary tra)t scoring. 

Holistic Scoring 

Holistic scoring of a writing sample Is based upon the reader's 
overall Impression of the effectiveness of a piece of writing. 
Papers are scored by, trained raters on a numerical scale, 
usually a four-point scale. Once the vvriting samples are col- 
lected, the raters or scorers sort the samples Into four stacks, 
relating the quality of the essay only to other papers in the 
group rather than to a predetermined example of "good" writ- 
ing. Papers are typically read by two raters, and the scores 
they assign a writing sample are summed into a total score. If 
there is a discrepancy of two score points, the score is recon- 
ciled by yet a third reader/rater. 



Primary Trait Scoring 

Primary trait scoring focuses on a specific rhetorical character-^ 
istic or trait of a given piece of writing. It Is based qn^'the 
premises that all writing is done In terms of a specific audi- 
ence and that writing. If successful, will have the desired 
effect on that audience, Lloyd-Jones (1977) Identlfirs two 
goals of primary trait scoring: (1) to define what segment of 
discourse will be evaluated (e,g., explanatory, expressive, per- 
suasive), and (2) to train readers to render holistic judgments 
accordingly. A scoring guide for primary trait analysis may 
consist of the exercise Itself; a description of the rhetorical 
traits of the writing; an Interpretation of the exercise indicat- 
ing how each element In the task is expected to affect the 
student; an Interpretation of how the situation of the exercise 
Is related to the primary trait; sample papers that are repre- 
sentative of each score point; and a discussion of why each 
sample paper was scored as It was (McCready and Melton 
1981). One difference between holistic and primary trait bcqx- 
-Hig telhatwit h p rim a ry-tr a it sc oring , stu dents*-ftaper^ afe-t>efng- 
measured against external criteria, whereas with holistic scor- 
ing, papers are compared with one another. 



Analytical Scoring 

If primary trait scoring . a situation-specific analysis of writ- 
ing, analytical scoring is a thorough, tralt-by-tralt analysis. The 
Identified traits reflect those components of a writing sample 
that are considered Important to any piece of writing In any 
context, DIederich (1974), the originator of analytical scoring, 
for example^ has Identified eight common traits: Ideas, organi- 
zation, wording, flavor (tone), usage, punctCiatlon, spelling, and 
handwriting. Others may use traits more general such as con- 
tent, organiiation, focus and support, and mechanics. If enough 
comppnonts.are analyzed, this scoring procedure can provide a 
comprehensive picture of writing performance. However, the 
components need to be explicit and well defined so that the 
raters understand and agree upon the basis for making judg- 
ments about the writing sample. 

In relating these scoring approaches to classroom applica- 
tions, Spandel (1981) observes that holistic scoring offers a 
broad base for a discussion of what makes a piece of writing 
generally good or bad. Analytical scoring can take this discus- 
sion one step further by identifying those traits of components 
that make a piece of writing effective. And, by being situation- 
specific, primary trait scoring focuBes on the importance of 
audience to a piece of writing. 

laauea and Problems 

Essential to the quality of assessment and the value of scoring 
procedures used are the reliability and validity of the scores 
generated by the assessment. Specifically, the scoring criteria 
should be applicable uniformly within a rating session and 
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from one rating session to another. Furthermore, these ratings 
should correlate with other measures of student writing. Even 
if the assessment instrument Is reliable and valid, spurious 
scores can-result from the development of poor exercises, poor 
test administration or environment, or poor scoring procedures 
(Stigglns, Northwest Regional Educational Laboratpry 1980). 
Scorers must understand and agree upon the criteria applied 
to a piece of writing. Thus, Inadequate training of scorers may. 
also influence or skew the results of an assessment. 

The choice of topic (or "prompt") to be written is another - 
factor that may Influence scores, Students may write more 
erthusiastically on some topics than on others, resulting in 
better quality writing. A student's background and prior knowl- 
edge will also affect the "expertise" a strident brings to a piece 
of writing. Or, depending upon the studtir.i's interpretation of Q 
writing prompt, he or she may write persuasive discourse in 
response to a prompt Intended for expository discourse. 

Tlm^and cost are two other factors that may influence the 
decision for large scale writing assessment. Stigglns (North- 
west Regional Educational Laboratory 1980) separates cost 
factors into those that are developmental and those that are 
.administrative. Developmental costs will vary depending on 
whether a previously designed assessment Instrument is used' 
or a new one developed. If one is to forego the expense of 
Constructing a new scoring instrument, expense will still be 
Incurred for the securinf reviewing, evaluating, and selecting 
of appropriate exercises and scoring guides that do exist. 
Administrative costs Involve those associated with test admin- 
istration, the selection of test administrators, the distribution 
of materials, and tho collection of test meterlals. 

Then there are the scoring costs—the time required to trein 
■raters and the time required to rate papers, According to data 
collected by Ouellmalz (1 982), the training time for holistic and 
primary trait scoring averages two to four hours, and for ana- 
lytical scoring avereges six to eight hours. Test reuse Is another 

/ Cost factor. Stigglns (Northwest Regional Educational Labora- 
/ tory 1980) states that "with Indirect essessment, the high cost 
of test Construction can be amortized over repeated adminis- 
tration and the recur'ring scoring costs are very loyv. However, 

, with direct assessment, although the Initial developmerit costs 
are low. the scoring costs remain high with each use." In a 
:1982 dissertation Bauer compared the reliabilities and the 
cost-efficiencies of these three methods of direct assessment. 
Based on her results, Bauer concluded that the analytical 
method was the moot reliable and the holistic method was 
the most cost-efficient in grading a large number of essays 
(see ED 225 171). 

A 1979 study by Fredrick Identified some of the problems 
that stated have encountered with their writing assessment 

I endeavors: (1 ) arrangements for a place large enough and suit- 
^Ne enough for the scoring, (2) coordinating release time for 
teachers to act as scorers^ (3) adhering to a rigid time schedule 
during the scoring session, (4) not enough time or money, and 
(5) finding or designing a variety of writing skill tasks. Her 
survey concluded with a list of recommendations to others, 
who are planning a writing assessment, some of which are 
as follows: | 

><JomLu[flle_w!jtinfl object^^^ andiocua, resea rch aue&tioas, 

before the assessment, 

• use language arts specialists to advise on content and to 
react to iterrja prior to pilot testing, 

• clarify traits to be measured, 

• Include clear and conplse directions, and 

• use actual, parformence on practical writing, such as 
messages, letters, forms and so forth, Instead of the 
proofreading type of assessment found in most multiple 
choice tests. 

Trends in Writing Assessment 

A national study conducted In 1981 by McCready and Melton 
collected date from 42 st/ate departments of eiiucAtion. Of tht» 
. 22 states using a writing sample, most of the states Indicated 
that they used holistic scoring, procedures, with three states 



using primary trait techniques, one using analytical, and three 
states using both hollstics and analytic scoring. In fact, when 
comparing their study with the earlier study by Fredrick (1979), 
McCready and Melton found a change in preference from 
either holistic or primary trait scoring to a use of holistic and 
analytical methods, which appeared to offer a broader base for 
determining basic competencies in wiriting and assessing edu- 
cational progress. 

The May 1 984 Issue of CAPTRENDS, published by the Cen- 
ter for Performance Assessment, reveals diverse environments 
for the solicitation of writing samples. Some states used 
untlmed writing samples, while others set a 25-minute limit. 
Some states allowed students to revise their initial drafts, 
while yet another state offered less skilled writers a number of 
prewriting suggestions to help them get started. 

Large scale writing assessments are useful, but complex.' 
This digest has attempted to identify a few of the issues and 
problems that need to be addressed in such an endeavor. 
However, as Spandel and Stigglns conclude in their booklet, 
D/r6ct Measures of Writing Skills: Issues and Applications. 
Reyised Edition (1981), "There is not now, nor will there ever 
be, a single best way to assess writing skills. Each individual 
educational assessment and writing circumstance presents 
unique problems to the developer and use of writing tests. 
Therefore, great care must be taken In selecting the approach 
'land the methods to be used In each writing assessment. 
Methods used In one context to measure one state of relevant 
writing skill should not be generalized to other writing contexts 
without careful consideration of writing circumstances." 

Holly O'Donnell, ERIC/RCS 
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